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ABSTRACT 

Using a 6-weeX, test-retest design with 37 
third-grade students, the stability of the Reading Style Inventory 
(RSI) was assessed. The consistency of the recommended reading method 
interventions that derive from the RSI was also eKamined. The RSI 
consists of 52 items organized into two sections. The first section 
contains 14 three-option questions and 12 two-option questions 
addressing student preferences for physical and social aspects of the 
learning environment. The second section contains 26 two-option 
questions relating to students* preferred modality. In addition to 
the basic inventory question booklets and response sheets, there is 
also a diskette for scoring the inventories and generating summary 
descriptive data and intervention recommendations, output from the 
scoring diskette includes summary descriptors of the stuf'fc.it's 
identified preferences, instructional/curricular interventions rated 
as "highly recommended", "recommended", "acceptable", or "not 
recommended"; and recommended modifications to each listed 
intervention other than those "not recommended." Results of this 
study indicate that reliabilities for the perceptual modality scales 
approximated those reported in the RJil manual, but confirm the 
inventory's low reliability. Reading method recommendations and 
strategies recommended for implementing those methods showed 
considerable variability across the two administrations of the RSI. 
Questions are raised concerning the RSI's use for prescriptive 
purposes given the observed instability. One tables is included. 
(TJH) 
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Abstract 



The purpose of the present study was to assess the stability of the Reading Styles Inventory 
{RSI), and the consistency of the recommended raading method inten/entions that derive from the 
RSI. The study used a six-week test-re*cSt design with 34 ttiird grade students. Reliabliities for 
the perceptual modality scales approximated those reported in the manual, and confirm the 
inventory's low reliability. Reading method recommendations, and strategies recommended for 
implementing those methods, showed considerable variability across the two administrations of 
the RSI. Questions are raised concerning use of the RSI for prescriptive purposes given the 
observed instability. 
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I* is a well accepted axiom among educational practitioners that children learn in different 
ways, and that Instruction should be adapted to the learner's needs. However, translating the 
axiom into practice poses significant problems. From the broadest perspective, adjusting to 
indiv dual differences among learners requires an adequate definition and delimitation of the 
individual difference construct, adequate operationalization of that construct definition, 
development of reliable and valid instruments and/or procedures for assessing that construct, and 
development of appropriate Interventions that derive from the relationship between the 
Individual difference variable and learning processes. Failure to deal with any one of these issues 
adequately would lead to findings that are problematic at best. Problems in these areas are 
evident in the area of learning styles research. Educational research provides very limited 
support for the efficacy of adjustments to individual difference variables among learners, 
including adjustments to learning style (see Cronbach & Snow, 1977). Early ATI research has 
been criticized as yielding mixed results because the attributes investigated were too broadly 
defined {Driscoll, 1987). This may account for some of the mixed results reported by Cronbach 
and Snow (1977). However learning styles research suffers from much of the same problem of 
definition. What are the components of learning style? Is style a stable trait or a learned 
characteristic? Should instruction be matched, mismatched, or mixed matched to style? Despite 
the lack of substantive evidence for efficacy, many teachers remain committed to the notion of 
learning styles, and seek ways to measure learners' styles and adjust instruction accordingly. 

Fundamental to the learning styles-based education movement are the assumptions that 
individuals do diffor in meaningful ways in their style of learning; that an individual's learning 
style represents a relatively stable trai^; that this trait can be assessed using relatively simple 
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instruments: and that, once an individual's style is Icnown, educational interventions can be 
identified or developed which wiil enhance learning. It is important to note that accepting any or 
all of these assumptions does not mean that one need accept the current means of operatlonalizing 
learning style or its assessment. 

One of the most popular techniques for assessing style is the use of an inventory. Such 
inventories typically asl< learners to self-report on the manner in which they feel they learn 
best, or on the conditions under which they feel they learn best, or prefer to learn. The use of 
self-report techniques focusing on preferences assumes that such aspects of cognitive functioning 
are subject to conscious knowledge and accurately reportable. Given these assumptions, it is very 
disappointing to see virtually no communication or collaboration among researchers In learning 
s^;les and researchers in the area of metacognition. 

As a field, metacognition focuses on understanding the knowledge that individuals have of 
their cognitive processes, and how they regulate those processes. One aspect of metacognition, 
metacognitive knowledge, would seem to be particularly relevant to the use of self-report 
inventories to assess children's learning styles. Research suggests that metacognitive knowledge, 
although stable and statable, is also fallible and relatively late developing (Baker & Brown, 
1984; Brown & Palincsar. 1982; Garner, 1987; Markman, 1979; Myers & Paris. 1978). 
Using self-report inventories with young children would therefore assume a level of 
metacognition that is questionable at best among young learners. 

Developing interventions ba&*2d on preferences also assumes th?! learning preference and 
learning strength are synonymous. Rita Dunn argues that students taught through 
preference-matched techniques achieve at higher levels and have better attitudes, and that such 
increases mean, de facto, that one's prefe-ence is one's strength (Dunn. 1990, p. 15). There is 
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counter evidence, however, in that students having difficulty In schools will often opt for low 
structure activity or group work instead of structured individual activity If given the chance. A 
substantial body of research on instructional design would suggest that such students would profit 
from more structured teaching approaches. In this case, what studenis might prefer is 
mismatched to their instructional needs. 

Although data *;om the learning styles consortium at St. Joi.n's suggest that matching 
instruction to students' learning styles can have a powerful impact on achievement, there is also 
cause for caution. For example. In the technical manual for the Reading Styles Inventory, Marie 
Carbo reports average September to f^ay gains of from 1 .5 to 4.1 Grade Level Equivalents (GLE's) 
on standardized reading achievement tests as a result of matching reading instruction to student's 
reading style (Carbo. 1982. p. 5). The results span grades one to six (f^ay GLE's from 2.8 for 
grade 1 to 8.1 for grade 6), with a total sample of 24 students laoeled as "Remedial Reading 
Students". Although Interpretation of GLE's is problematic, a reasoned analysis would suggest that 
these reported gains are enormous. Given the tendency to report average effect sizes of .8 or above 
as large effects In meta-analyses, and the strong message that min,.nal Intervention should be 
expected to yield little or no benefit In Cronbach and Snow's (1977) treatment of ATI research 
{including learning styles), these results border on the miraculous. 

Although most Inventories address learning in general (eg. Kolb's LSI; Dunn, Dunn, & 
Price's LSI; NASSP's LSP; McCarthy's Format), one, the Reading Styles Inventory (Carbo, 
1982), is subject specific. In principle, giving a specific task context for student responses 
might lead to greater consistency of response to the inventory questions by the learner, and lead to 
the generation of more vaiid intervention recommendations. Specifying the context for 
self-report of learning strategies has been found to increase ti.e correspondence between 
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predicted or anticipated behavior and actual behavior in comprehension monitoring tasks with 
competent readers (ForllzzI & Clark, 1989). 

Marie Carbo has also suggested that once a learner's reading style is identified, the learner 
need not be reassessed for a period of up to three years. Such might be a reasonable 
recommendation if reading style were a stable trait, if learners had sufficient self-awarene« ot 
that style to respond to the instrument, and if the instrument were technically sound. However, 
data on the inventory citec in the technical manual are disappointing. Test-retest reliabilities on 
the various scales of the r.SI are reported only up to a six-week period. Even so, the reliabilities 
are somewhat low. If this lack of stability in categorization also yields variable and potentially 
contradictory Intervention recommendations, then the use cf the Inventory as a basis for 
instructional design and intervention is suspect. 

Given these questions, the purpose of this study is to assess the stability of the Reading Style 
Inventory over a six-week Interval, and to assess the consistency of the intervention 
recommendations from the RSI over the six-week interval. 

Method 

Dsslgn. This study used a traditional two assessment reliability design with a six-week 
interval between testings. 

Subjects. Subjects were 45 students in two third grade classrooms in a local public school 
system. Two students who were present at second testing were absent for the initial test, three 
students who had been absent for the initial testing were present for the second testing, two 
students moved between testings, and one new student entered school prior to the second testing. 



This meant that only partial data were available for eight students resulting In a final sample of 
37 students for whom complete data were available. 

Materials- The RSI consists of 52 items organized in two sections. The first section contains 
fourteen three-option questions and twelve two-option questions addressing student preferences 
for physical and social aspects of the learning environment. The second section contains 
twenty-six two-option questions relating to students' preferred modality. In addition to the basic 
inventory question booklets and response sheets, there is also a disltette for scoring the 
inventories and generating summary descriptive data and intervention recommendations. 

Procedures. Subjects were administered the Reading Styles inventory early in the school 
year (October), and again after a six week period (early November). The instrument was 
group-administered by the classroom teacher during a regularly scheduled reading class in 
accordance with procedures specified in the manual. 

Results 

Test-retest relia'/.litv. In order to calculate test-retest reliabilities, we needed to 
reconstruct scores for the sixteen various scales of the RSI. Relevant data for construction cf such 
scores are not available in the technical manual, nor are they provided with the output from the 
computer scoring diskettes. After solicitation of scoring information from the publisher (National 
Reading Styles Institute), we were provided with item numbers 5?nd keyed responses for the four 
perceptual modality scales, but were denied access to scoring information for the remaining 
twelve scales. The four modality scales (visual, auditory, tactual, and kinesthetic) consisted of R, 
7, 6, and 5 items respectively. These four scales, then, reflect 50% of the total items leaving the 
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remaining 26 items to be split among the otlier twelve scales. 

Table 1 provides test-retest reliabilities reported for each scale in the technical manual 
(3-week interval) and data obtained from the present study. As can be seen, reliabilities obtained 
In the present study for the four perceptual scales were comparable to those reported 'n 'he 
manual. A subsequent analysis of response consistency to the various items in each subscale 
indicated that there was a median of 29% response changes (range from 1 9% to 46%) overall on 
the perceptual scales items. These items were all two-option Items, so a shift In response would 
reflect a preference for "the opposite" type of intervention than selected on the first 
administration of the inventory. 

Intervention Recommendations. Output from the scoring diskette includes summary 
descriptors of the student's identified preferences; instructional/cur ricular interventions rated 
as "highly recommended", "recommended", "acceptab'e", or "not recommended"; and recommended 
modifications to each listed intervention other than those "not recommended." Although not 
specifically stated in the manual, it appears as though the initial intervention/curricular 
recommendations were based on matching the student's modality strength(s) with the focus of 
specific methodologies, while modifications to those recommendations were based on responses to 
the remaining 12 scales ( 26 items). There were 7 different methods recommended through the 
diskette scoring package, Language Experience, Whole Word, Individualized Method, Phonic 
Method, Carbo Recorded Book Method, the Fernald Method, and the Orton Gillingham method. Table 
2 presents a sample output with recommendations. 

In order to assess the consistency of recommended interventions, we chose to consider only 
those methods that were rated as "highly recommended" under the assumption that those methods 
were most closely matched to the student's identified reading style and would be the intervention 



of choice given availability of materials. Recommendations were then tabled for each student for 
the initial and second testings. The analysis revealed that only 60% of the methods that were rated 
as "highly recommended" after the initial testing were also rated as "highly recommended" after 
the second testing. 

We also tabled the recommended modifications to the "highly recommended" methods. This 
analysis indicated that only 55% of the modifications to the "highly recommended" methods from 
the first testing were also recommended as modifications for the same method after the second 
testing. 

Discussion 

Results from the present study relating to test-retest reliability of the RSI are consistent 
with data reported in the technical manual for the instrument. That suggests that there 's no 
reason to believe that subjects in this study showed any less stable level of response to inventory 
questions than would be anticipated from other samples of students. However, an analysis of the 
intervention recommendations and instructional modifications to those intervention 
recommendations reveals a great deal of inconsistency. This would suggest that either reading 
style is not a particularly stable construct {trait?), that the RSI is not sufficiently sound to 
detect stable aspects of the construct, that there is no dear link between inventory responses and 
intervention recommendations, or a combination of these factors. In any case, serious questions 
are raised as to how to match intervtntion to assessed style given such instability in both 
assessment and intervention recommendations. 

Despite these concerns, there are numerous advocates of lea-iing styles based approaches to 
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instruction, and ample reports in the literature of success stories following matching instruction 
to assessed style (see for exampie the October, 1990 issue of Educational Leadership.) While it 
may be difficult to question performance data that are reported, it is less clear to what the 
Improvement should be ascribed. Given the high level of Instability in "highly recommended" 
reading methods noted in the present study, it stands to reason that over the course of a relatively 
finite instructional period, a substantial proportion of students would either have to switch 
methods regularly to maintain a "highly recommended" match remain with a method that might 
not be "highly recommended" as a match. If efficacy claims rely on stable style-method matches, 
then these data cast doubt on that explanation. We could find no research studies that attempted an 
assess-match, roassess-rematch design to assess the extent of potential method changes. However 
it is interesting to speculate on what it would mean to observe improvements in each case. 

It would seem t^lat the logical next step in learning styles research shouH be to identify the 
potential loci of treatment potency. If true potency does not lie in stable style-method matches, 
then perhaps it resides in the ancillary adjustments to physical and social aspects of the learning 
environment. 
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Table 1 

Reliabilities for Scales of the RSI 



SCALE 



grade; 



MANUAL 
GRADE 4 



COMBINED 



OUHDATA 
GRADES 



SOUND .82 

UGHT .77 

TEMPERATURE .67 

DESIGN .80 

MOTIVATION .68 

PERSISTENCE .65 

RESPCNSI5ILTY .68 

STRUCTURE .76 

SOCKXDGtCAL .69 

INTAKE .76 

TIMEOFD/-Y .72 

MOBILITY .67 



.73 
.68 
.66 
.67 
.72 
.64 
.70 
.70 
.55 
.59 
.63 
.73 



.72 
.69 
.65 
.71 
.70 
.63 

.72 
.67 
.67 
.66 
.76 



PERCEPTUAL SUBSCALES: 
AUDITORY .79 
VISUAL .83 
TACTUAL .78 
KINESTHETIC .75 



.75 
.71 
.68 
.74 



.74 
.77 
.69 
.75 



.59 
.71 
.69 
.62 
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